The IAFOR Journal of Education 


Volume III - Issue I - Winter 2015 


Control Grouped Pedagogical Experiment to Test 
the Performance of Second-generation Web Maps and 
the Traditional Maps at the University of Debrecen 


Daniel Balia, Marianna Zichar, Judit Boda, Tibor Jozsef Novak 


31 



The IAFOR Journal of Education 


Volume III - Issue I - Winter 2015 


Abstract 

Almost every component of the information society is influenced by elements built on 
communication technology. Learning also tends to be related to the dynamic usage of computers. 
Nowadays, a number of applications (online or offline) are also available that engage large 
groups of potential users and simultaneously provide a virtual environment to facilitate learning. 

This study introduces the self-developed interactive blind map teaching-examining e-learning 
system of the University of Debrecen. Results of testing the system with a control group are also 
presented.Both experimental and control groups of students were required to sita test of 
topographic knowledge following a semester of study. The pass mark for the test was 80%. The 
experimental group used the new digital environment to study, while the control group prepared 
for their exam using paper maps in the traditional way. The key research questions addressed by 
the study were to determine whether exam results obtained by the group using the ‘digital’ 
method better than those of the control's; and if there were a difference between the exam 
performances of the two groups, was this statistically significant and, therefore, likely to occur in 
other similar scenarios? 

Keywords: Web map; e-learning; education of geography. 
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Introduction 

Both bachelor and master students majoring in geography have to significantly develop their 
knowledge of topography during their university studies. One of the most fundamental courses 
for bachelor students is Topography. This establishes knowledge of physical and social 
geography during the first year. In the next semesters (Table 1) studies focus on specific 
geographic entities and are confined to a smaller geographical region. For example, in the case of 
Physical Geography of Hungary students have to be able to locate lands, rivers, mountains on a 
blind map during the exam. Passing a topography test is a prerequisite of entering several 
colloquiums, so insufficient knowledge prevents the students even registering for examinations. 

Earlier experience shows that students often incorrectly position entities in the blind map. One 
reason may related to insufficient cartographic and map reading skills. However, maps used for 
practise are usually created by students who, often, do not pay enough attention to map projection. 
If the projections of the maps used for preparation and in the exam are different, then it is 
unlikely that students will be able to correctly locate entities using the blind map. The maximum 
number of allowable mistakes to pass an exam depends on the course. Nevertheless, the level of 
aggregation in the blind maps is not standardized. For example, in the case of physical geography 
tests, recognition of the location of a river or a mountain may help to place additional entities, 
while in social geography tests (with settlements, buffer zones) not only the physical objects but 
also cadastral and other borders have to be taken into consideration. 

Traditionally, tests are paper-based in all cases. This requires a huge quantity of paper. There are 
approximately 100 students enrolled on each of the five years and a topography test may be 
retaken twice in a semester (Table 1). Thus, currently some 10 000 sheets are printed and stored. 
This volume continues to grow from year to year and The Institute of Earth Sciences has no 
remaining capacity to archive this information. 

Table 1. List of courses with a topography test as a prerequisite 


Course name 

Semester 

Degree 

Number of 

tests 

Estimated 
number of 
students 

Estimated 
number of tests 

Topography 

I. 

BSc 

18 

100 

1800 

The principles of social geography I. 

I. 

BSc 

1 

100 

100 

The principles of economical geography. 

III. 

BSc 

2 

100 

200 

Physical geography of Hungary 

IV. 

BSc 

7 

100 

700 

Social geography of Hungary 

IV. 

BSc 

2 

100 

200 

Physical geography of Europe 

IV. 

BSc 

3 

100 

300 

Social geography of Europe 

IV. 

BSc 

1 

100 

100 

Physical and social geography of the 

Earth I. 

V. 

BSc 

2 

100 

200 

Physical and social geography of the 

Earth II. 

V. 

BSc 

1 

100 

100 

Conservation of geo heritage 

Autumn 

MSc 

1 

15 

15 

Total 





3715 
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As a result of the above issues, a suggestion was made to design and develop an easy-to-use blind 
map based application that could also support the process of learning and teaching. Additionally, 
this could provide measurable data for further methodological research. 

The main aim of application development was to support the work of the Institute of Earth 
Sciences, and to support teaching-learning processes using online interactive environments 
(Zichar, 2011). However, an additional benefit is that educators no longer have to correct tests 
manually, because the application performs it in a few seconds and also suggests an appropriate 
grade. 


Literature Review 


E-Learning 

Holmes and Gardner (2006) refers to e-learning, as the use of new multimedia technologies and 
the internet to improve the quality of learning, by facilitating access to resources and services as 
well as remote exchange and collaboration. This can take place completely online in virtual 
environments, or, using a combination of virtual and face-to-face environments. E-Learning has 
potential to influence education positively. It provides great opportunities for both the educators 
and learners to enrich their educational experience (Albon, 2005; Al-Harbi, 2011). In addition, 
supported by the openness and flexibility of the internet, e-learning provides the opportunity for 
teaching and learning transactions with alrnos unlimited information, and is also relatively 
unaffected by pressures of time and constraints of distance (Holmes & Gardner, 2006; Oya & 
Uchida, 2013). 

This section provides an overview of the key advantages and disadvantages of e-learning based 
on the work of Ozuorcun & Tabak (2012). 

Advantages of e-learning: 

• Class work can be scheduled around personal and professional work, resulting in flexible 
learning. 

• Learners may have the option to select learning materials that meets their level of 
knowledge and interest. 

• Learners can study whenever and wherever they have access to a computer and the 
internet. 

• Self-paced learning modules allow learners to work at their own pace. 

• Successful completion of online or computer-based courses builds self-knowledge and 
self-confidence and encourages students to take responsibility of their learning. 

E-learning uses a special teaching/learning model that has limitations too (Mohammedi et ah, 

2010 ). 

• E-learning requires less contact between the students, and also between the teachers. 

• Knowledge about general computer usage and the e-leaming system are prerequisites. 

• Learning requires a computer and internet access as well. 

• E-learning implies different teaching/learning processes and some teachers may have less 
experience and acquaintance about this processes. 

• Lack of face-to-face interaction with a teacher can imply negative impact. 
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Educational web maps in Hungary 

The use of digital maps in education has increased significantly in Hungary (Dombovari et ah, 
2010). Maps are also used in areas such as history, biology, natural history, social sciences and 
medical epidemiology. This trend appears, of course, in our everyday life, when we navigate 
while travelling or even walking around large shopping centres (Gede et ah, 2013; Zentai & 
Kovacs, 2013). Considering such widespread use, learning to read and interpret maps can be 
regarded to be a general life skill (Jones et ah, 2004). According to Neumann (2008), web maps 
are fundamentally digital maps which are optimized and displayed on a monitor and appropriate 
for web claims. It is widely accepted that application of these maps is of service regardless of 
their generation and type. (Kraak, 2001; Zentai & Dombovari, 2005; Plewe, 2007). 

One of the first web map applications was “The Blind Mouse”, published in 2005 and used to 
support education nationally throughout Hungary (see Figure 1). The program developed at the 
Eotvos Lorand University is appropriate for checking topographic knowledge. In the course of 
using this program, one must place the objects appearing on the screen as accurately as possible. 
The level of difficulty (primary school, high school, and university) can be set in advance. It is 
important to note that turning on the layers (hydrography, relief and border) clearly helps 
orientation. A new version of Blind Mouse, “Blind Mouse 3D” was published in 2010 
(Dombovari & Gede, 2010). Although the program remained essentially unchanged, the 
underlying web map has been updated from a second generation digital map to a fourth 
generation virtual globe platform. Many elementary and high schools in the country include this 
application in their education because of its free availability and innovative characteristics. 



Figure 1. "The Blind Mouse" 

The efforts to design a self-developed interactive blind map teaching-examining e-learning 
system began in 2010 at the University of Debrecen. The new digital environment for practising, 
the prearranged control maps and the opportunity to edit the online curriculum are valuable 
services for numerous students majored in geography. The new e-leaming application based on J 
1.0 was planned to be used in education and exams as well (Balia, 2012). 

Methodology and methods 


The implementation steps 

Implementation was in two stages. First, the layers of the digital base map are determined. This is 
followed by creating the teaching maps, including the natural conservation areas and landscape 
districts in Hungary, using ArcMap 10. The second stage concerned completing programming 
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tasks, creating the database and testing the application after establishing the web server. Selection 
requirements for development technologies included open access and capability for locating 
geographical objects on a blind map; presenting the attributes of selected objects; round-the-clock 
availability; and with secure password-protected login user privilege administration facilities. 


Layers of the digital base map 

Development tool 

Statistical Analysis 

Nature conservation areas represented 
by points 

PHP (PHP Hypertext 

Pre-processor) 

Analysis of variance 

Lakes and rivers represented by lines 

Flash 

F-test 

Nature conservation landscapes 
represented by polygons 

HTML (Hyper Text 
Markup Language) 

• two-sample t-test 

• Welch-test 


The program J1.2 


The online user interface of the program, named J 1.2 (supported browser: Mozilla Firefox) is 
published on a webpage (http://webgis.web44.net/program.php?tipus=oktato) . This distinguishes 
two user types: students and teachers. Teachers may specify new exams by providing 
administrative data, pass-mark (percentage) and fault tolerances (in pixels). The database can be 
extended to include the locations of new objects at any time. The student user interface is 
designed for exam practise as well as for formally sitting examinations. Student scores are stored 
and may be retrieved for individuals and for entire exams. With the help of the interactive 
mapquiz interface, the geographic objects stored in the database may be browsed and inspected 
(see Figure 2). The aim of the application is to check and evaluate student knowledge concerning 
the location of named geographic entities; this following earlier study using the same application. 
Although level of difficulty is not assigned to the items, teachers are free to select the items to be 
used in an examination. This, therefore, provides a means for creating tests with different levels 
of difficulty (Balia, 2013). 
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Figure 2. The user interface of "J 1.2" 

The web maps were designed to be used for tests issued on a course concerning the conservation 
of geoheritage. The database stores information on nature conservation and landscape protection 
areas. This comprises some 181 records (with attribute data such as id, name, rank, description). 
Of these 143 records concern nature conservation areas and the remaining 38 are of landscape 
protection areas. During the collection of core data a key intention was to develop student 
geographical knowledge, with a focus on contemporary (live) and physical (inanimate) values of 
geology in Hungary. 

Creation of the experimental and control group 

The study was based on rwo groups of master students, registered for the course “Conservation of 
Geoheritage”. From representative sampling it was apparent that the groups were similar (not 
significantly different) in their composition. The two groups are identified by X and Y notation. 
The group Y completed a paper test in traditional way in December 2012. One year later every 
student from the group Y was assigned a unique account for online application, after collecting 
data necessary for registration. Since our objective was to compare the results of the two groups, 
both groups completed the same test. 


Results 

Test results of the control group and experimental group 

The group X and Y had 5 minutes to complete the test. During this time they had to place 10 
objects on a blind map as accurately as they could. A pass mark of 80% was required to pass the 
test. If objects placed on a blind map had point geometry, it was also necessary to also specify a 
fault tolerance for distance. Here a tolerance of 1 cm was set as the the maximum allowable 
distance between the true object location and the position specified by the student. Using these 
criteria, 54 % of students in the control group (Y) passed the test with marks of 80% or greater. 

The most common errors were due to inaccurate placing. In 3 or 4 cases, the placement might 
have been accepted if fault tolerance had been increased slightly. 

Contrastingly 77% of the digital test group (X) passed with marks greateror equalt to the 80% 
threshold. On further examination findings revealed that most group X students either scored 
highly or had only failed to pass by a narrow margin. The overall average score for group X was 
8.62, compared to 7.31 for control group Y (see Figure 4). 
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Test results of the control group and experimental group 
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Figure 3. Test results of the control group and experimental group 

The result of the hypothesis study 

The experimental group consisted of 13 students, just like the control group. The two groups can 
be considered homogenous according to their main properties. The students had to solve the same 
test in both groups after a two-month-long practicing period. We are referring the results of the 
experimental group by X, and that of the control group by Y. The Figure 4 shows that both data 
lines are of normal distribution despite of the few element numbers. The experiment of the 
control group begins with examining the variance, than it continues with 2 samples T-test or 
Welch-test depending on the results (Loki & Demeter, 2009; Falus & Olle, 2000). 
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Figure 4. Histograms and plots of group X and Y 

The computation highlights that the variance value of the examined samples do not differ from 
each other significantly, which means that the two-sample t-test can be performed to compare the 
results (Guilford, J.P., 1965; Peers, I., 1996). 

Table 2. demonstrates the result of the two-sample t-test f’=1.901 (the degree of freedom is 24). 
The probability belonging to this value is 0.69, which means a level of 93.1%. Since the 
calculated t-value is less than the appropriate value from the table (t ta bi e : =2,064) we have to claim 
that the difference of the arithmetic means calculated from the results of the two examined 
samples was random and cannot be explained only by introducing the new methodology. 

To sum up, the performance of the students in the experimental group was really better than in 
the control group, but unfortunately the difference is not significant enough to prove that the 
better results are related to the new method and that its applications would lead to better results 
with the given probability. 


Table 2.The result of the two examined samples calculated 



Levene's Test for 
Equality of 
Variances 

t-test for Equality of Means 

F 

Sig. 

t 

df 

Sig. (2- 
tailed) 

Mean 

Difference 

Std. Error 
Difference 

95% Confidence 
Interval of the 
Difference 

Lower 

Upper 

Result 

Equal variances 
assumed 

1.889 

0.182 

1.901 

24 

0.069 

1.308 

0.688 

-0.112 

2.728 

Equal variances 
not assumed 



1.901 

21.041 

0.071 

1.308 

0.688 

-0.123 

2.738 


Conclusion 

This paper presents a self-developed interactive blind map teaching-examining e-learning 
application of the University of Debrecen, and the test results with a control group. 

One of the greatest advantages of using e-leaming is that it gives university students greater 
flexibility in time and place to enhance their knowledge. Moreover, e-learning systems increase 
motivation; users are more likely to adopt and use e-leaming if they deem it makes the process of 
studying easier (Giraudo & Peruch, 1992; Golledge, 1992; Ndubisi, 2006; Al-Harbi, 2011). This 
study had to address certain limitations. 
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First, e-learning had not yet been widely adopted in the University Of Debrecen at the time of the 
study. Secondly, the current study relied on onecohort that may not be completely representative 
of the wider student body, thus unavoidably affecting the confidence with which findings could 
be generalized to other circumstances. Thirdly, findings are similar to that of Rittschof & 
Kulhavy’s (1998). It was also found that a high degree of familiarity with the real regional 
geography is necessary in order that students acquire a consistent and undistorted knowledge of 
their geographical environment. 

The following conclusions concern key two aspects of the J1.2 teaching program: 

Firstly, despite non-significant differences between test scores of control and experimental groups, 
results of the pedagogical experiment were promising. This was because of the good exam results 
achieved with a digital platform, and the experience gained during preparation and delivery of the 
exam. Although statistically unproven, the direction of findings remains consistent with the view 
that digital maps enhanced learning experience, and this should be perhaps explored further in 
more detailed studies drawn on larger samples. It also appeared that computer supported 
teaching and examination could be more efficiently delivered for the ‘digital’ experimental group 
than for the control one. 

Secondly, concerning the production of digital learning blind maps, GIS software was used to 
create maps based on a selected area, where geographical entities were represented by using 
different layers (e.g. hydrography, country border, conservation areas, etc.). GIS and map 
properties, such as aggregation level, map projection and resolution and the number and type of 
layers used, all greatly influence the user experience and dynamic of interaction with the 
mapping application. Thus it is important not to overcrowd maps with large numbers of 
overlapping elements. This not only undermines the visual experience, but is also an obstacle to 
processes of understanding and studying. These issues are concerns for the future development of 
this system. Immediate priorites are to focus on development of the user interface to enhance 
interactivity and overall experience. At the same time it is hoped that this application will be 
applied in other courses (such as topography examinations), thereby further disseminating it as an 
effective and safe environment to support the work of both students and educators. 

Future directions 

In addition to introducing the mapping application into other areas of education, the following 
enhancements are also planned. 

• Designing a more developing a more attractive graphical user interface 

• Creating and applying web maps of third and fourth generation 

• Extension of the application to be able to manage line and polygon objects too 

• Opportunity for student feedback 

• Accuracy tests in maps with different scale 

• Increasing interactivity 

• Hypothesis study 
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